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(54) Method and device for control and delivery of digitally compressed visual data in a 
heterogeneous communication network 

(57) A method and apparatus automatically manip- 
ulates a digitally compressed video bitstream such that 
the result is compatible with different bandwidth and 
error resilience requirements at a receiving point in a 
heterogeneous communication network. The method 
and device provide for analysis of bitstream header 
information to determine its relevant coding syntax 
parameters. Feedback from the receiving point on the 
network is then used to identify error robustness 
requirements for an intermediate bitstream. The bit- 
stream is then manipulated by an error robustness reg- 
ulator to achieve error protection in the network. The 
bitrate requirements of the network establish a selection 
of relevant and irrelevant bits from the bitstream The 
present method and device include identification of irrel- 
evant and low priority video bits; intentional introduction 
of packet losses to reduce bitrate while maintaining vis- 
ual quality; use of replacement tags for irrelevant and 
low priority components of the bitstream for compatible 
decoding by a wide array of decoders; and insertion of 
^synchronization tags and translation of predictively 
coded video packet bits for the improvement of error 
resilience. 
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Description 

Field of the Invention 

[0001] The present invention relates to automati- 
cally managing the delivery of digitally compressed vis- 
ual data, and more particularly, to a device and method 
for controlling the bitrate and error robustness of digit- 
ally compressed video bitstreams in a heterogeneous 
communications network. 

Background of the Invention 

[0002] With the advent of new communication sys- 
tems, it has become possible to communicate digital 
visual information and data in addition to just voice serv- 
ices efficiently. Such new communication systems com- 
prise both wireline and wireless networks. In the context 
of wireline networks, packet and circuit switched com- 
munication systems have been developed in both the 
private and public domains for wide and local area cov- 
erage. As part of packet-based communications, a sig- 
nificant amount of data is exchanged between local 
area networks, such as those which are part of a corpo- 
rate infrastructure, and wide area networks, such as the 
public Internet. Services to guarantee Quality of Service 
(QoS) have emerged in this realm to support visual and 
multimedia communication. 

[0003] New wireless systems have also been devel- 
oped to support the transmission of information at 
higher bitrates than previously possible. These wireless 
systems conform to international cellular communica- 
tion standards, and consist of digital communication air- 
interface standards using Time Division Multiple Access 
(TDMA) and Code Division Multiple Access (CDMA). An 
example of widely deployed TDMA communication is 
the Global System for Mobile communication (GSM). 
CDMA forms the basis for the third generation cellular 
communication standard IMT-2000. 
[0004] Most of these communication systems (wire- 
line and wireless) have been designed to allow for the 
exchange of a maximum amount of information 
between users connected within the infrastructure of 
each system and those using devices outside that infra- 
structure. For example, IMT-2000 protocols can support 
data communication from wireless terminals to termi- 
nals residing on packet switched wireline systems. 
Moreover, within these communication systems, termi- 
nals, or clients, may connect to other terminals, acting 
as servers, at different bitrates depending on their loca- 
tion in each system. This leads to the concept of heter- 
ogeneous communication networks, which are 
networks consisting of diverse or dissimilar connections 
yet often communicating similar information. 
[0005] Problems arise when considering the com- 
munication of visual information in heterogeneous net- 
works. A first problematic issue which arises is that 
visual information, and, in particular, digitally com- 



pressed video signals, require substantially more band- 
width than comparable voice signals compressed over 
similar time intervals. Transmitting visual information 
between systems designed with different nominal trans- 

5 mission bitrates presents some difficulty because of 
delays that result from these incompatible bitrates. In 
addition, varying error characteristics of the different 
communication systems in a heterogeneous network 
also present a problem for the transmission of visual 

w information because it is very difficult to mask errors in 
the visual space. 

[0006] These problems do not alone make up the 
difficulties with communicating visual information over 
diverse systems. Services designed to exploit the com- 

75 munication of visual information in heterogeneous net- 
works must, necessarily, rely on the existence of 
digitally compressed video. Therefore, not only creating 
new content, but exploiting existing content is a primary 
focus for the further development and success of these 

20 services. For one-way visual applications, there is 
already a substantial amount of digitally encoded con- 
tent for training and entertainment that could be deliv- 
ered to mobile subscribers on a wireless network, or to 
clients connected at dissimilar rates on a wireline net- 

25 work. Existing, or "legacy" video material is primarily 
encoded using the ISO MPEG-1 and MPEG-2 stand- 
ards, although legacy content also exists in the form of 
the ITU-T H.261 and, to a lesser extent, H.263 stand- 
ards. This material can have high value to new multime- 

30 dia services, while at the same time lead to the 
acceptance and use of new visual coding standards 
such as ISO MPEG-4. 

[0007] One problem with the delivery of legacy data 
over the low bandwidth wired and wireless networks is 

35 that the original encoded bandwidth typically exceeds 
the capabilities of the network, assuming that there is a 
maximum delay restriction on the delivery of that data. 
For example, some wireless mobile systems may have 
multimedia terminals supporting MPEG-1 decoding, but 

40 the channel rates to the multimedia enabled mobile ter- 
minals may be as low as 64Kbps. In this case, the 
MPEG-1 material, encoded at 1.5Mbps, would require 
substantial buffering delay time before playback could 
begin. Unless most of the sequence was to be down- 

45 loaded first, stalling would occur, as the decoder's buffer 
would empty much faster than it would fill. It is also 
unlikely that a mobile terminal would be able to provide 
sufficient memory for buffering long sequences. A 
requirement exists, therefore, to successfully manipu- 

50 late this type of high bitrate data to a lower rate such 
that it is compatible with a low bandwidth connection on 
the network. 

[0008] As a result of the arrival of new wired and 
wireless communication systems with the capacity to 
55 transmit and exchange visual information, there is a sig- 
nificant need for a method and device designed to man- 
age the visual content being delivered over these 
networks. In particular, technology is needed to auto- 
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matically manage the bitrate and error robustness of 
pre-existing digital video bitstreams so that they can be 
delivered in a compatible form to users at arbitrary 
nodes in a heterogeneous network. 

Summary o f the Invention 

[0009] Briefly, therefore, this invention provides a 
method and apparatus for changing a digitally com- 
pressed video bitstream at a source point in a network, 
in an automatic way, such that the resulting bitstream is 
compatible with different bandwidth and error resilience 
requirements at a receiving point in a heterogeneous 
communication network. The novel method consists of 
analyzing the header information of the original bit- 
stream to determine its coding parameters. Feedback 
from the receiving point on the network is then used to 
determine error properties of the network. The band- 
width and error resilience requirements establish a pri- 
oritization and selection of relevant and irrelevant bits 
from the original bitstream, of which the relevant bits are 
manipulated by a robustness regulator to achieve error 
protection in the network. The principal inventive com- 
ponents of the method include identification of irrelevant 
and low priority video bits, intentional introduction of 
packet losses to reduce bitrate, use of replacement tags 
for irrelevant and low priority components of the bit- 
stream, and insertion of ^synchronization tags for the 
improvement of error resilience. 

Brief Description of the Drawings 

[0010] 

FIG. 1 is a block diagram of a bitstream regulation 
system operative to manipulate an incoming bit- 
stream and output a bitstream compatible with a 
receiving terminal in accordance with an embodi- 
ment of the present invention. 
FIG. 2 is a generalized flow diagram of a preferred 
embodiment of a method for regulating a bitstream 
encoded with scalable information to produce a bit- 
stream compatible with a receiving terminal in 
accordance with the present invention. 
FIGS. 3A, 3B and 3C illustratively provide examples 
of insertion of resynchronization tags into a bit- 
stream and translation of differentially encoded 
components within a video packet to support robust 
decoding. 

FIGS. 4A and 4B illustratively provide examples of 
insertion of replacement tags into a bitstream after 
bitrate regulation to preserve the decoded quality at 
a compatible receiving terminal. 
FIG. 5 illustrates a block diagram of a preferred 
embodiment of a device for manipulating bits in a 
bitstream of encoded video data to produce a bit- 
stream compatible with a receiving terminal in 
accordance with the present invention. 



Detailed Descriptio n of the Preferred Embodiments 

[001 1 ] The present invention is more fully described 
with reference to FIGS. 1 - 5. FIG. 1 illustrates a block 

5 diagram 100 of a visual bitstream regulation method 
which defines a control system operated in the manner 
described herein. The input bitstream 101 is passed 
through a header analysis stage 103 which extracts 
information about the type of standard used to create 

10 the bitstream. This information is passed through the 
standard type identification stage 105, which results in 
the assignment of control parameters in the main bit- 
stream control 107 which is coupled to accept input 
parameters describing the receiving terminal's capabili- 

75 ties 1 09. These steps are designed to set up the system 
for proper regulation and compatible delivery of the bit- 
stream from the sender to the receiving terminal. In par- 
ticular, identification of the original visual coding 
standard used to encode the bitstream coupled with 

20 information about the capabilities of the receiving termi- 
nal determines the extent to which the bitstream must 
be manipulated before compatible communication is 
possible. 

[001 2] The output of the main bitstream control step 

25 is coupled to the robustness regulation stage 111. In 
this stage, the bitrate-regulated bitstream is modified to 
provide a bitstream which is compatible with the error 
characteristics of the channel connecting the source 
and receiving terminals. This robustness regulation is 

30 dependent on the standard type of the original bitstream 
identified in 105. The regulation is accomplished 
through the packetization of bits in the coded frames of 
visual information in such a way that errors may be 
detected, localized and concealed by a robust decoder. 

35 The use of resynchronization tags 1 1 3 is applied for vis- 
ual bitstreams conforming to standards which support 
sub-frame resynchronization. The robustness regula- 
tion step manipulates the bitstream such that independ- 
ent decoding is possible for each segment of bits 

40 delimited by the inserted resynchronization tags. In the 
case of predictively coded information, this information 
is replaced with equivalent information utilizing only pre- 
dictors within the same segment. This stage is referred 
to as video packet translation 115. For example, this 

45 would apply to the replacement of codewords for motion 
vector differences computed across resynchronization 
segments with motion vector differences computed from 
information in the same segment. 
[0013] The output of the video packet translation 

so step is directed to the bitrate regulation stage 1 1 9 which 
is coupled to receive intentional packet loss information 
117 and, additionally, to discard irrelevant bits 121 in 
order to reduce the bitstream bitrate. The purpose of the 
bitrate regulation stage is to adjust the bitrate of the 

55 input bitstream so that the delivered stream is compati- 
ble with the capabilities of the receiving terminal and its 
delivery channel. This is accomplished through manipu- 
lation and removal of bits in the incoming bitstream. The 
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matically manage the bitrate and error robustness of 
pre-existing digital video bitstreams so that they can be 
delivered in a compatible form to users at arbitrary 
nodes in a heterogeneous network. 

Summary of the Invention 

[0009] Briefly, therefore, this invention provides a 
method and apparatus for changing a digitally com- 
pressed video bitstream at a source point in a network, 
in an automatic way, such that the resulting bitstream is 
compatible with different bandwidth and error resilience 
requirements at a receiving point in a heterogeneous 
communication network The novel method consists of 
analyzing the header information of the original bit- 
stream to determine its coding parameters. Feedback 
from the receiving point on the network is then used to 
determine error properties of the network. The band- 
width and error resilience requirements establish a pri- 
oritization and selection of relevant and irrelevant bits 
from the original bitstream, of which the relevant bits are 
manipulated by a robustness regulator to achieve error 
protection in the network. The principal inventive com- 
ponents of the method include identification of irrelevant 
and low priority video bits, intentional introduction of 
packet losses to reduce bitrate, use of replacement tags 
for irrelevant and low priority components of the bit- 
stream, and insertion of resynchronization tags for the 
improvement of error resilience. 

Brief Description of the Drawings 

[0010] 

FIG. 1 is a block diagram of a bitstream regulation 
system operative to manipulate an incoming bit- 
stream and output a bitstream compatible with a 
receiving terminal in accordance with an embodi- 
ment of the present invention. 
FIG. 2 is a generalized flow diagram of a preferred 
embodiment of a method for regulating a bitstream 
encoded with scalable information to produce a bit- 
stream compatible with a receiving terminal in 
accordance with the present invention. 
FIGS. 3A, 3B and 3C illustratively provide examples 
of insertion of resynchronization tags into a bit- 
stream and translation of differentially encoded 
components within a video packet to support robust 
decoding. 

FIGS. 4A and 4B illustratively provide examples of 
insertion of replacement tags into a bitstream after 
bitrate regulation to preserve the decoded quality at 
a compatible receiving terminal. 
FIG. 5 illustrates a block diagram of a preferred 
embodiment of a device for manipulating bits in a 
bitstream of encoded video data to produce a bit- 
stream compatible with a receiving terminal in 
accordance with the present invention. 



Detailed Description of the Preferred Embodiments 

[001 1 ] The present invention is more fully described 
with reference to FIGS. 1 - 5. FIG. 1 illustrates a block 

5 diagram 100 of a visual bitstream regulation method 
which defines a control system operated in the manner 
described herein. The input bitstream 101 is passed 
through a header analysis stage 103 which extracts 
information about the type of standard used to create 

10 the bitstream. This information is passed through the 
standard type identification stage 105, which results in 
the assignment of control parameters in the main bit- 
stream control 107 which is coupled to accept input 
parameters describing the receiving terminal's capabili- 

is ties 1 09. These steps are designed to set up the system 
for proper regulation and compatible delivery of the bit- 
stream from the sender to the receiving terminal. In par- 
ticular, identification of the original visual coding 
standard used to encode the bitstream coupled with 

20 information about the capabilities of the receiving termi- 
nal determines the extent to which the bitstream must 
be manipulated before compatible communication is 
possible. 

[001 2] The output of the main bitstream control step 

25 is coupled to the robustness regulation stage 111. In 
this stage, the bitrate-regulated bitstream is modified to 
provide a bitstream which is compatible with the error 
characteristics of the channel connecting the source 
and receiving terminals. This robustness regulation is 

30 dependent on the standard type of the original bitstream 
identified in 105. The regulation is accomplished 
through the packetization of bits in the coded frames of 
visual information in such a way that errors may be 
detected, localized and concealed by a robust decoder. 

35 The use of resynchronization tags 1 1 3 is applied for vis- 
ual bitstreams conforming to standards which support 
sub-frame resynchronization. The robustness regula- 
tion step manipulates the bitstream such that independ- 
ent decoding is possible for each segment of bits 

40 delimited by the inserted resynchronization tags. In the 
case of predictively coded information, this information 
is replaced with equivalent information utilizing only pre- 
dictors within the same segment. This stage is referred 
to as video packet translation 115. For example, this 

45 would apply to the replacement of codewords for motion 
vector differences computed across resynchronization 
segments with motion vector differences computed from 
information in the same segment. 
[0013] The output of the video packet translation 

so step is directed to the bitrate regulation stage 1 1 9 which 
is coupled to receive intentional packet loss information 
117 and, additionally, to discard irrelevant bits 121 in 
order to reduce the bitstream bitrate. The purpose of the 
bitrate regulation stage is to adjust the bitrate of the 

55 input bitstream so that the delivered stream is compati- 
ble with the capabilities of the receiving terminal and its 
delivery channel. This is accomplished through manipu- 
lation and removal of bits in the incoming bitstream. The 
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stream in order to guarantee accurate decoding at a 
receiving terminal. The introduction of replacement 
codeword bits 231 accomplishes this task in the next 
step of the method. In a preferred embodiment as 
shown by the flow diagram 200, ^synchronization 5 
packets are dropped in step 223. FIGS. 4A and 4B fur- 
ther describe the inserting of replacement information 
for two scenarios of dropped MPEG-4 video packet 
information in accordance with this preferred embodi- 
ment. In the first example 401 shown in FIG. 4A, bits 10 
associated with the texture coding part of a video packet 
are dropped by the bitrate regulator 21 7 in FIG. 2. Por- 
tion 401a of the first example identified an original data 
partitioned video packet bitstream; and portion 401b 
represents video packet bitstream with replacement 15 
tags. The bits dropped, which are delimited by a unique 
motion marker, are associated with the spatial predic- 
tion error codewords (CBPY. DQ, DCT). which decode 
to values applied to a spatial portion of a video frame 
after motion compensation has been applied. This infor- 2 
mation is the lowest priority component of an MPEG-4 
video packet. While this example illustratively provides 
details for the syntax of MPEG-4, the identical concept 
applies to the video syntax of the MPEG-2 standard. In 
example 401 , replacement bits are inserted into the bit- 2 t 
stream in the form of code words for zero valued predic- 
tion error coefficients (CBPY'^Q'.DCT). These 
codewords require negligible bitrate in relation to the 
bitrate consumed by the dropped bits. A more extreme 
case of the same concept is illustrated in example 403 3L 
shown in FIG. 4B, where data partitioning is not present 
in the incoming bitstream. Portion 403a of FIG. 4B rep- 
resents an original combined motion texture video 
packet bitstream; and portion 403b represents video 
packet bitstream with replacement tags. In this case, all 35 
of the bits associated with the payload of the video 
packet are dropped. The video packet header remains 
and is used to indicate the bounding macroblock infor- 
mation for the video packet The replacement bits used 
are the codewords for combined motion and texture 40 
information for blocks containing no motion and no pre- 
diction error information (CBP 1 ). This is referred to as 
"NOT CODED" codewords in example 403. Again, as in 
example 401 , the bitrate consumed by the replacement 
bits is negligible in comparison to the bitrate consumed 4S 
by the original video packet under consideration. 
[0025] Continuing with the description of the pre- 
ferred embodiment of the present method described by 
FIG 2, a bitrate regulated bitstream is output at 233 from 
the bitrate regulation stage. The output bitstream is so 
delivered to the heterogeneous network 237 for subse- 
quent decoding at the receiving terminal 239. 
[0026] FIG. 5 illustrates a diagram of a device 500 
for manipulating a digitally compressed video bitstream 
to produce an output bitstream compatible with the 55 
bandwidth and error resilience requirements of hetero- 
geneous communications network. The device 500 
comprises a header analysis unit for determining a 



standard type and a syntax used to compress the input 
bitstream 501 coupled to a main bitstream controller 
505. A header analysis unit 503 is coupled to receive 
input from a memory module holding a list of predeter- 
mined standard header codes 504 enabling the analysis 
unit to compute the present standard and syntax by 
exhaustive matching of header bits. The main bitstream 
controller 505 is a means for discarding selected bits 
from the input bitstream on the basis of the syntax 
determined in the unit 503. The main bitstream control- 
ler 505 comprises a robustness regulation module 509, 
a bitrate regulation module 51 1, and a replacement tag 
generator 515. 

[0027] The robustness regulation module 509 is a 
means for inserting predetermined bits into the bit- 
stream to improve the error robust localization and 
decoding of the bitstream in accordance with the stand- 
ard type and syntax determined in the unit 503. The 
robustness regulation module 509 also translates any 
predictively coded information into codewords relying 
only on bits within the present video packet in accord- 
ance with inserted ^synchronization markers. Opera- 
tion of the robustness regulation module 509 is reliant 
on signals from the header analysis unit 503 and feed- 
back from the receiving terminal 507. which are both 
coupled to input to the main bitstream controller 505 
containing those modules. 

[0028] The output of the robustness regulation 
module 509 is coupled to input to the bitrate regulation 
module 51 1 , which is a means to discard selected bits 
513 from the bitstream on the basis of the standard type 
and the syntax determined in the unit 503. The bitrate 
regulation module 509 is a means for dropping low pri- 
ority bits associated with frames in an enhancement 
layer of a scalable bitstream, and means for discarding 
packets of low priority bits associated with segments in 
a packet of video bits delimited by ^synchronization 
markers or picture headers. The bitrate regulation mod- 
ule 509 also contains memory for recording historical 
information of the packets dropped in a previous frame 
which is used to compute the next packets to drop. This 
enables avoiding repetitive removal of identical spatial 
areas of a scene. 

[0029] The bitrate regulation module 51 1 produces 
an output which is compatible with the capabilities of a 
receiving terminal as indicated by the feedback signal 
507 coupled to input to the main bitstream controller 
505. The output of the bitrate regulation module 51 1 is 
coupled to the input of the replacement tag generator 
515, which is a means for inserting predetermined bits 
to improve the decoding of the bitstream in accordance 
with the standard type and syntax determined in the unit 
503. In particular, the replacement tag generator 515 
replaces bits dropped by the bitrate regulation module 
509 with short code words designed to preserve timing 
and spatial information in any decoder such that it will 
not suffer from unrecoverable errors. 
[0030] The output of the main bitstream controller 
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505 is coupled to a heterogeneous network 519, which 
in turn delivers a resultant bitstream 517 to a robust 
decoder at a receiving terminal 521 . The device 500 is 
capable of manipulating bitstreams generated by any 
video coding standard. Depending on the degree of 5 
flexibility of the particular standard, varying degrees of 
functionality in terms of bitrate regulation and error 
robustness are achieved. In general, the most recent 
standards, H.263 and MPEG-4, can provide for the 
most fine control of these bitstream attributes by the 10 
device 500. 

[0031] Although exemplary embodiments are 
described above, it will be obvious to those skilled in the 
art that many modifications and alterations to this 
method and device for bitstream control and delivery is 
may be made without departing from the invention. 
Accordingly, it is intended that all such modifications 
and alterations be included within the spirit and scope of 
the invention as defined in the following claims. 

20 

Claims 

1. A method for manipulating bits in a bitstream of 
encoded video data, said method comprising the 
steps of: 25 

determining a standard type and a syntax used 
to compress the bitstream; 
inserting predetermined bits into the bitstream 
and translating predictively coded information 30 
resulting in an intermediate bitstream with 
improvement of error robust localization and 
decoding of the bitstream in accordance with 
the standard type and the syntax; and 
discarding selected bits from the intermediate 35 
bitstream based on the standard type and the 
syntax used to compress the bitstream result- 
ing in an output bitstream having a bitrate com- 
patible with that of a robust receiving terminal. 

40 

2. The method of Claim 1 wherein the step of deter- 
mining standard type and a syntax used to com- 
press the bitstream further comprises the step of: 



so 

3. The method of Claim 1 wherein the step of inserting 
predetermined bits and translating predictively 
coded information further comprises the step of: 



4. The method of Claim 1 wherein said step of dis- 
carding selected bits further comprises the step of: 

discarding low priority bits associated with 
frames in an enhancement layer of a scalable 
bitstream. 

5. The method of Claim 1 wherein said step of dis- 
carding selected bits further comprises the step of: 

discarding packets of low priority bits associ- 
ated with segments in a packet of video bits 
delimited by resynchronization markers or pic- 
ture headers. 

6. The method of Claim 5 wherein the step of discard- 
ing packets of low priority bits further comprises the 
step of: 

selecting packets to discard on the basis of his- 
torical information of packets dropped in a pre- 
vious frame to avoid repetitive removal of 
identical spatial areas of a scene. 

7. A device for manipulating bits in a bitstream of 
encoded video data, said device comprising: 

means for determining a standard type and a 
syntax used to compress the bitstream; 
means for inserting predetermined bits into the 
bitstream and translating predictively coded 
information resulting in an intermediate bit- 
stream with improvement of error robust locali- 
zation and decoding of the bitstream in 
accordance with the standard type and the syn- 
tax; and 

means for discarding selected bits from the 
intermediate bitstream based on the standard 
type and the syntax used to compress the bit- 
stream resulting in an output bitstream having 
a bitrate compatible with that of a robust receiv- 
ing terminal. 

8. The device of Claim 7 wherein said means for 
determining a standard type and a syntax used to 
compress the bitstream further comprises: 

means for reading header bits and comparing 
said bits to a predetermined list of standard 
headers to identify a present standard type and 
syntax used to compress the bitstream. 

9. The device of Claim 7 wherein said means for 
inserting predetermined bits and translating predic- 
tively coded information further comprises: 

means for inserting resynchronization markers 
and translating differentially encoded motion 



inserting resynchronization markers; and 55 
translating differentially encoded motion vec- 
tors and associated texture data parameters to 
limit predictive coding to within a video packet. 



reading header bits; and 45 
comparing said bits to a predetermined list of 
standard headers to identify a present standard 
type and syntax used to compress the bit- 
stream. 
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vectors and associated texture data parame- 
ters to limit predictive coding to within a video 
packet. 

10. The device of Claim 7 wherein said means for dis- s 
carding selected bits further comprising: 

means for discarding low priority bits associ- 
ated with frames in an enhancement layer of a 
scalable bitstream. 10 

11. The device of Claim 7 wherein said means for dis- 
carding selected bits further comprising: 

means for discarding packets of low priority bits is 
associated with segments in a packet of video 
bits delimited by ^synchronization markers or 
picture headers. 

12. The device of Claim 1 1 wherein the means for dis- 20 
carding packets of low priority bits further com- 
prises: 

means for selecting packets to discard based 
on historical information of packets dropped in 25 
a previous frame to avoid repetitive removal of 
identical spatial areas of a scene. 
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